A number of useful and popular search engines attempt to maintain full text indexes of the World Wide Web. For example, search engines are available from AltaVista, Excite, HotBot, Infoseek, Lycos and Northern Light. However, searching the Web can still be a slow and tedious process. Limitations of the search services have led to the introduction of meta search engines. A meta search engine searches the Web by making requests to multiple search engines such as AltaVista or Infoseek. The primary advantages of current meta search engines are the ability to combine the results of multiple search engines and the ability to provide a consistent user interface for searching these engines. Experimental results show that the major search engines index a relatively small amount of the Web and that combining the results of multiple engines can therefore return many documents that would otherwise not be found.
A number of meta search engines are currently available. Some of the most popular ones are MetaCrawler, Inference Find, SavvySearch, Fusion, ProFusion, Highway 61, Mamma, Quarterdeck WebCompass, Symantec Internet FastFind, and ForeFront WebSeeker.
The principle motivation behind the basic text meta search capabilities of the meta search engine of this invention was the poor precision, limited coverage, limited availability, limited user interfaces, and out of date databases of the major Web search engines. More specifically, the diverse nature of the Web and the focus of the Web search engines on handling relatively simple queries very quickly leads to search results often having poor precision. Additionally, the practice of “search engine spamming” has become popular, whereby users add possibly unrelated keywords to their pages in order to alter the ranking of their pages. The relevance of a particular hit is often obvious only after waiting for the page to load and finding the query term(s) in the page.
Experience with using different search engines suggests that the coverage of the individual engines was relatively low, i.e. searching with a second engine would often return several documents which were not returned by the first engine. It has been suggested that AltaVista limits the number of pages indexed per domain, and that each search engine has a different strategy for selecting pages to index. Experimental results confirm that the coverage of any one search engine is very limited.
In addition, due to search engine and/or network difficulties, the engine which responds the quickest varies over time. It is possible to add a number of features which enhance usability of the search engines. Centralized search engine databases are always out of date. There is a time lag between the time when new information is made available and the time that it is indexed.